docs: orthogonal persistence concept page by marc0olo · Pull Request #16 · dfinity/developer-docs

marc0olo · 2026-03-16T17:02:34Z

Summary

Concept page explaining ICP's orthogonal persistence model, following Diataxis "explanation" principles (what/why, no how-to code):

What orthogonal persistence means and the mental model shift from traditional backends
Two memory regions: heap (4/6 GiB) and stable memory (up to 500 GiB, subject to subnet storage budget)
Motoko's true orthogonal persistence (persistent actor, transient var) — the only ICP language with transparent persistence
Rust's explicit stable structures approach — contrasted as not orthogonal, full developer control
The dangerous pre_upgrade heap serialization anti-pattern and skip_pre_upgrade recovery
Trade-offs table differentiating Motoko and Rust use cases
Comparison with traditional database-backed backends
Further reading: IC Internals blog post, Stellarator deep dive, YouTube short

Implementation code examples are intentionally omitted — they belong in the data-persistence how-to guide (stub brief updated to cover them).

Also adds a "Shared storage budget" bullet to the network-overview subnets section.

Sync recommendation

Informed by dfinity/portal — persistence sections, and stable-memory icskill

Copilot

Pull request overview

Adds a new Concepts page explaining ICP’s orthogonal persistence model (how canister state persists across calls and upgrades) and contrasts Motoko’s automatic persistence with Rust’s explicit stable-structures approach.

Changes:

Replaces the placeholder page with an end-to-end explanation of heap vs. stable memory.
Adds Motoko (persistent actor, transient var) and Rust (ic-stable-structures, MemoryManager, StableBTreeMap) examples.
Adds guidance on why heap-serialization upgrade patterns are risky, plus trade-off/comparison tables and next-step links.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

+
+A separate, dedicated memory region provided by the Internet Computer runtime. Its sole purpose is to survive canister upgrades.
+
+- **Size limit:** Hundreds of GB (bounded by the subnet storage limit, approximately 500 GB)


+- **`#[init]` and `#[post_upgrade]`** handlers must be defined. Stable structures auto-restore, so `post_upgrade` only needs to reinitialize transient state (timers, caches)
+- **No `pre_upgrade` serialization needed** -- data is already in stable memory


+
+| | Heap memory | Stable memory |
+|---|---|---|
+| **Size limit** | 4 GB (wasm32) / 6 GB (wasm64) | Hundreds of GB |


marc0olo · 2026-03-16T18:14:20Z

Review: Orthogonal Persistence

Must fix

Memory size inconsistency — "approximately 500 GB": The page says "approximately 500 GB" in the stable memory section and "Hundreds of GB" in the trade-offs table. Other docs pages (e.g., docs/concepts/canisters.md) use "up to 500 GiB". Align to "up to 500 GiB" in both locations. GiB vs GB is a meaningful distinction (500 GiB ≈ 537 GB).
"Bricked" claim is overstated: The "dangerous pattern" section says when pre_upgrade traps, "the canister is bricked — the upgrade fails and the data cannot be recovered." The IC interface spec provides a skip_pre_upgrade flag specifically for this recovery scenario (though it risks data loss). Soften to something like: "the upgrade fails, and recovery requires the skip_pre_upgrade flag, which may result in data loss."

Suggestions

#[init] and #[post_upgrade] "must be defined": Technically, stable structures auto-restore without these hooks — they're only needed to reinitialize transient state (timers, caches). The page already notes this in the parenthetical. Consider softening to "should be defined" since the icskill says "the canister may silently reset state." The current "must" is defensible as good practice but slightly stronger than the reality.
Heap memory "wiped" framing for Rust: The page says heap is "Historically wiped on canister upgrade (Rust)." The word "historically" implies this changed — it didn't for Rust (raw heap data is still wiped on upgrade). Consider: "Wiped on canister upgrade (Rust) — use stable structures to persist data."
Trade-offs table "Use case" row: The table says heap is for "Caches, temporary computation" and stable memory for "All persistent application data." In Motoko with persistent actor, the runtime persists heap data transparently — so the heap IS used for persistent data in Motoko. The table is Rust-centric. Consider adding a note or qualifier.

Verified

All 3 internal links resolve to existing files (../guides/backends/data-persistence.md, ../languages/rust/stable-structures.md, ../guides/canister-management/lifecycle.md)
Motoko code verified against .sources/motoko/ and .sources/motoko-core/: persistent actor, transient var, Map.empty, Map.add, Nat.compare all confirmed
Rust stable structures pattern verified against icskill: MemoryManager, VirtualMemory, StableBTreeMap, DefaultMemoryImpl, MemoryId, thread_local! pattern all match
Heap memory limits "4 GB for wasm32, 6 GB for wasm64" — correct
No CLI commands or step-by-step procedures — properly stays in concept/explanation territory (Diataxis compliance)
No dfx references, no .mdx/JSX, plain markdown only
Frontmatter complete with icskills: [stable-memory]
Content brief fully covered: heap persistence, stable memory, Motoko auto-persistence, Rust stable structures, trade-offs, traditional backend comparison
Upstream attribution comment present
Code examples are under 30 lines each

- Fix memory sizes to use GiB consistently (500 GiB, 4 GiB, 6 GiB) - Add subnet shared storage budget note with link to subnet selection - Soften "bricked" to mention skip_pre_upgrade recovery flag - Change #[init]/#[post_upgrade] from "must" to "should" be defined - Remove "historically" from Rust heap wipe description - Make trade-offs table language-aware (Motoko uses heap for all data) - Add shared storage budget bullet to network-overview subnets section

marc0olo · 2026-03-17T09:51:12Z

Feedback addressed:

Fixed "approximately 500 GB" → "up to 500 GiB" in stable memory section, aligned trade-offs table
Also fixed heap memory sizes to use GiB consistently (4 GiB / 6 GiB)
Added note about subnet shared storage budget with link to subnet selection guide
Softened "bricked" → mentions skip_pre_upgrade flag for recovery (with data loss risk)
Changed #[init]/#[post_upgrade] from "must be defined" to "should be defined" with explanation of why
Removed "historically" from Rust heap wipe — now reads "Wiped on canister upgrade (Rust) — use stable structures to persist data"
Made trade-offs table language-aware: heap use case now says "All data in Motoko persistent actor; caches and temporary computation in Rust"
Added "Shared storage budget" bullet to network-overview "What subnets mean for developers" section

- Remove Motoko and Rust code examples (belong in data-persistence guide) - Clarify that orthogonal persistence is Motoko-exclusive; Rust uses explicit stable structures (not orthogonal) - Replace implementation details with conceptual descriptions + links to the data-persistence how-to guide - Add "Further reading" section with Medium articles and YouTube short - Update data-persistence stub brief to cover content moved from here - Add subnet shared storage budget to network-overview

marc0olo · 2026-03-17T10:04:52Z

Feedback addressed (restructure):

Diataxis compliance: Removed all implementation code (Motoko and Rust examples). This page is now a pure concept/explanation page. Code examples and implementation details will live in the data-persistence how-to guide (stub brief updated).
Orthogonal persistence is Motoko-exclusive: Clarified that only Motoko delivers true orthogonal persistence. Rust section now explicitly states "This is not orthogonal persistence" and contrasts the explicit approach.
Further reading: Added three external resources — IC Internals blog post, Stellarator Part 2 deep dive, and YouTube short explainer.
Learn Hub: No relevant Learn Hub articles exist in the inventory for this topic.
All previous feedback items (GiB units, skip_pre_upgrade, should vs must, heap wipe wording, trade-offs table) remain applied.

- Intro now distinguishes Motoko (transparent) vs Rust (explicit) persistence - Comparison table uses precise "500 GiB per canister, subject to subnet storage budget" instead of vague "up to subnet limit" - Align heap memory to "4 GiB (wasm32) / 6 GiB (wasm64)" in canisters.md and cycles-costs.md to match orthogonal-persistence.md and icskills

marc0olo · 2026-03-17T12:34:45Z

Feedback addressed:

Intro now distinguishes Motoko (fully transparent persistence) vs Rust (explicit stable structures) upfront, instead of implying all persistence is automatic
Comparison table: "up to subnet limit" → "up to 500 GiB per canister, subject to subnet storage budget"
Cross-site heap memory alignment: updated canisters.md and cycles-costs.md to "4 GiB (wasm32) / 6 GiB (wasm64)", matching this page and the icskills reference

marc0olo added 2 commits March 16, 2026 17:50

docs: orthogonal persistence concept page

68b838d

fix: add Storable mention and improve Rust stable structures description

611fbe0

marc0olo requested a review from Copilot March 16, 2026 17:09

Copilot started reviewing on behalf of marc0olo March 16, 2026 17:10 View session

Copilot AI reviewed Mar 16, 2026

View reviewed changes

marc0olo merged commit e062737 into main Mar 17, 2026
1 check passed

marc0olo deleted the docs/concepts-orthogonal-persistence branch March 17, 2026 12:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: orthogonal persistence concept page#16

docs: orthogonal persistence concept page#16
marc0olo merged 5 commits into
mainfrom
docs/concepts-orthogonal-persistence

marc0olo commented Mar 16, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

marc0olo commented Mar 16, 2026

Uh oh!

marc0olo commented Mar 17, 2026

Uh oh!

marc0olo commented Mar 17, 2026

Uh oh!

Uh oh!

marc0olo commented Mar 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		A separate, dedicated memory region provided by the Internet Computer runtime. Its sole purpose is to survive canister upgrades.

		- Size limit: Hundreds of GB (bounded by the subnet storage limit, approximately 500 GB)

		- `#[init]` and `#[post_upgrade]` handlers must be defined. Stable structures auto-restore, so `post_upgrade` only needs to reinitialize transient state (timers, caches)
		- No `pre_upgrade` serialization needed -- data is already in stable memory

Conversation

marc0olo commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Sync recommendation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

marc0olo commented Mar 16, 2026

Review: Orthogonal Persistence

Must fix

Suggestions

Verified

Uh oh!

marc0olo commented Mar 17, 2026

Uh oh!

marc0olo commented Mar 17, 2026

Uh oh!

Uh oh!

marc0olo commented Mar 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

marc0olo commented Mar 16, 2026 •

edited

Loading